Implementation of an Online Intraoperative Assessment of Technical Performance for Surgical Trainees.

INTRODUCTION: Assessment of surgical resident technical performance is an integral component of any surgical training program. Timely assessment delivered in a structured format is a critical step to enhance technical skills, but residents often report that the quality and quantity of timely feedback received is lacking. Moreover, the absence of written feedback with specificity can allow residents to seemingly progress in their operative milestones as a junior resident, but struggle as they progress into their postgraduate year 3 and above. We therefore designed and implemented a web-based intraoperative assessment tool and corresponding summary "dashboard" to facilitate real-time assessment and documentation of technical performance. MATERIALS AND METHODS: A web form was designed leveraging a cloud computing platform and implementing a modified Ottawa Surgical Competency Operating Room Evaluation instrument; this included additional, procedure-specific criteria for select operations. A link to this was provided to residents via email and to all surgical faculty as a Quick Response code. Residents open and complete a portion of the form on a smartphone, then relinquish the device to an attending surgeon who then completes and submits the assessment. The data are then transferred to a secure web-based reporting interface; each resident (together with a faculty advisor) can then access and review all completed assessments. RESULTS: The Assessment form was activated in June 2021 and formally introduced to all residents in July 2021, with residents required to complete at least one assessment per month. Residents with less predictable access to operative procedures (night float or Intensive Care Unit) were exempted from the requirement on those months. To date a total of 559 assessments have been completed for operations performed by 56 trainees, supervised by 122 surgical faculty and senior trainees. The mean number of procedures assessed per resident was 10.0 and the mean number per assessor was 4.6. Resident initiation of Intraoperative Assessments has increased since the tool was introduced and scores for technical and nontechnical performance reliably differentiate residents by seniority. CONCLUSIONS: This novel system demonstrates that an online, resident-initiated technical assessment tool is feasible to implement and scale. This model's requirement that the attending enter performance ratings into the trainee's electronic device ensures that feedback is delivered directly to the trainee. Whether this aspect of our assessment ensures more direct and specific (and therefore potentially actionable) feedback is a focus for future study. Our use of commercial cloud computing services should permit cost-effective adoption of similar systems at other training programs.

How We Do It: Remediation Pathways in a Surgical Simulation Curriculum for Competency Improvement.

OBJECTIVE: Validated assessment of procedural knowledge and skills with formative remediation is a foundational part of achieving surgical competency. High-fidelity simulation programs provide a unique area to assess resident proficiency and independence, as well as to assist in identifying residents in need of further practice. While several studies have validated the use of simulation to attain proficiency of specific technical skills, few have validated remediation pathways for their trainees objectively. In this descriptive analysis, we review 2 remediation pathways within our simulation training curricula and how these are used in assessments of resident proficiency. MATERIALS AND METHODS: Two methods of remediation were formulated for use in high-fidelity simulation labs. One remediation pathway was a summative process, where ultimate judgment of resident competency was assessed through intra-operative assessments of a holistic skill set. The second remediation pathway was a formative "coaching" process, where feedback is given at several intervals along the pathway towards a specific technical skills competence. All general surgery residents are enrolled in the longitudinal, simulation curricula. RESULTS: Approximately one-third of surgical residents entered into a remediation pathway for either of the high-fidelity simulation curricula. Both residents and faculty expressed support for the summative and formative remediation pathways as constructed. Residents who entered remediation pathways believed it was a beneficial exercise, and the most common feedback was that remediation principles should be expanded to all residents. Interestingly, faculty demonstrated stronger support for the formative coaching feedback model than the summative assessment model. CONCLUSIONS: Through the complementary use of both formative and summative remediation pathways, resident competence can be enriched in a constructive, nonpunitive method for self-directed performance improvement. Both trainees and faculty express high satisfaction with programs explicitly organized to ensure that skills are rated through a standardized process.

Can a Structured, Video-Based Cadaver Curriculum Demonstrating Proficiency Enhance Resident Operative Autonomy?

INTRODUCTION: Autonomy is of foremost concern in the current era of surgical residency, and it is especially important to trainees when considering their surgical education. Factors impacting trainee independence include the restriction of clinical work hours and the development of advanced minimally invasive techniques such as robotics, which requires separate technical education outside of conventional surgical education. Moreover, when residents are left to learn fundamental exposures via their clinical experience alone, they run the risk of not being exposed to some fundamental skills based on case volume and type. The Department of Surgery at Emory University developed a cadaver-based simulation curriculum to standardize exposure to fundamental operative skills and enhance proficiency outside the operating room, with the larger aim of improving resident autonomy. METHODS: Residents were assigned to small groups led by a chief resident with an even distribution of postgraduate year (PGY) levels. Each group participated in core surgical exposures and fundamental maneuvers on a cadaver over a 6-hour session. Residents were tested on skills according to their PGY level about 1 month after the course. Testing included recitation of the skill in an oral boards format, highlighting major steps, followed by performance of the skill. All steps were video-recorded with no resident identifiers. These were reviewed by 2 independent, blinded faculty examiners who assigned proficiency grades to each resident video. RESULTS: Three hundred and thirty-three individual procedure evaluations were done over the 5-year period. Senior residents (PGY3-5) had 86% pass rate while junior residents (PGY1-2) had 70% pass rate. Overall, 21% of residents failed to achieve competence in their assigned skills. Junior residents were less likely to achieve competence compared to senior residents. Faculty graders had improved congruence in grading as the course progressed through the 5 years. The most recent 2 years had >80% congruence in faculty grading compared to less than 50% congruence in the first 2 years. 81% of attendings agreed this course positively influenced the granting of autonomy in the operating room. CONCLUSIONS: A cadaveric skills course focused on fundamental maneuvers with objective confirmation of achieving competency is a viable adjunct to clinical operative experience. Video-recorded evaluation, of these fundamental skills improved both resident and attending confidence in trainee operative skill.

Cadaver-Based Simulation Increases Resident Confidence, Initial Exposure to Fundamental Techniques, and May Augment Operative Autonomy.

OBJECTIVE: Surgical simulation is an important adjunct in surgical education. The majority of operative procedures can be simplified to core components. This study aimed to quantify a cadaver-based simulation course utility in improving exposure to fundamental maneuvers, resident and attending confidence in trainee capability, and if this led to earlier operative independence. DESIGN: A list of fundamental surgical procedures was established by a faculty panel. Residents were assigned to a group led by a chief resident. Residents performed skills on cadavers appropriate for PGY level. A video-recorded examination where they narrated and demonstrated a task independently was then graded by attendings using standardized rubrics. Participants completed surveys regarding improvements in knowledge and confidence. SETTING: The course was conducted at the Emory University School of Medicine and the T3 Laboratories in Atlanta, GA. PARTICIPANTS: A total of 133 residents and 41 attendings participated in the course. 133 (100%) participating residents and 32 (78%) attendings completed surveys. RESULTS: Resident confidence in completing the assigned skill independently increased from 3 (2-3) to 4 (3-4), p < 0.01. Residents stated that a median of 40% (interquartile range: 20%-60%) of procedures were performed for the first time in the course, and the same number had been performed only in the course. The percentage of skills attendings believed residents could perform independently increased from 40% (40%-60%) to 60% (60%->80%), p < 0.04. Attendings were more likely to grant autonomy in the operating room after this exercise (4 [3-5]). CONCLUSIONS: A cadaveric skills course focused on fundamental maneuvers with objective confirmation of success is a viable adjunct to clinical operative experience. Residents were formally exposed to fundamental surgical maneuvers earlier as a result of this course. This activity improved both resident and attending confidence in trainee operative skill, resulting in increased attending willingness to grant a higher level of autonomy in the operating room.

An analysis of tobacco industry marketing to lesbian, gay, bisexual, and transgender (LGBT) populations: strategies for mainstream tobacco control and prevention.

Research on adult tobacco use consistently shows a higher prevalence among lesbian/gay/bisexual/transgender (LGBT) populations than among the general population-reasons why are largely unknown, and counterstrategies are critical. Tobacco industry marketing, uncovered when the Master Settlement Agreement (MSA) forced companies to share its internal documents, provided insight. The American Legacy Foundation uncovered the industry campaign Project SCUM (Sub-Culture Urban Marketing) aimed at gays and the homeless. The formerly secret documents revealed specific marketing toward LGBT, whose rates increased when the MSA banned youth (but not other population) advertising. The industry reaches out to LGBT persons through direct and indirect advertising, community outreach, and sponsorships. Messages to LGBT have been relatively absent from advertising until recently, creating receptivity to such overtures. Reducing LGBT smoking rates is a public health challenge that will require exceeding the sense of validation tobacco advertising has created in LGBT communities.

Building capacity for success: the Tobacco Technical Assistance Consortium.

The Master Settlement Agreement with the tobacco industry has infused resources into tobacco control and prevention, which has helped fund tobacco control programs at the national, state, and local levels. Many tobacco control programs face challenges in areas such as strategic planning, fiscal operations, advocacy, and personnel management. To provide technical assistance to these programs, the Robert Wood Johnson Foundation, the American Legacy Foundation, and the American Cancer Society combined resources to fund the Tobacco Technical Assistance Consortium (TTAC). TTAG, housed at the Rollins School of Public Health at Emory University, provides support for tobacco control programs and matches requests for technical assistance with consultants who have the appropriate expertise. TTAC is a valuable addition to tobacco control and prevention resources and is a viable component within a school of public health.